PROBABILISTIC METHODOLOGY FOR RECORD LINKAGE DETERMINING ROBUSTNESS OF WEIGHTS By:

Over time, the world population has developed a desire to research their ancestoral linage. Many resources have been identified to aid an individual in genealogical research. In the United States, one of the greatest resources for researching genealogy is census records. Census records allow a genealogical researcher to track individuals over time, broadening the scope of information one can acquire about an individual. Dr. Halbert Dunn first presented the concept of record linkage in 1946 to describe the process, which joins two separate pieces of information for a particular individual or family [Dunn 1946]. Later, Fellegi and Sunter [1969] built upon Dunn’s foundations by establishing a probabilistic mathematical approach to record linkage. Probabilistic methods for record linkage have been developed to mimic the decision process of genealogists and researchers. An automated probabilistic approach allows the researcher to conduct many different types of searches within seconds. Following an automated search a list of record matches (links) as well as potential matches (links) with information necessary to further explore each potential record pair can be made. This enables a researcher to compile large numbers of records in a fraction of the time it would take to process manually. Probabilistic methods have been applied to determine the feasibility of linking persons across multiple census years. With a set of known weights to use in the record linkage process, one would eliminate the need to examine a large number of records manually. This paper uses probabilistic methods to link census records from the 1910 and the 1920 census indices to illustrate the benefits of an automated record linkage approach. CENSUS INDICES Since before 1850, census records have provided information regarding one’s demographic and personal information. Census indices contain a subset of the information found on a census page. In addition to omitting some information, census indices only include records for the head of household and individuals that differ in last name from the head of household [Szucs 2001]. Because of the limited information available in a census index, the defining demographics to be used in record linkage are likewise limited. The subset of information found in a census index is as follows: surname, given name (sometimes a middle name or initial is present in the field for given name), age at the time of census, gender, race, country of origin, state of residence, county of residence, district of residence, and information about the census page the information is located. When linking census records from any time period, it is important to account for discrepancies between censuses and failings inherent in the censuses. Because records from 1910 and 1920 have been used herein, issues relating to these census years will be presented. In 1918, at the end of World War I many Eastern European boundaries were realigned, changing the “place of origin” for many immigrants in the United States. For instance, an individual listing their “place of origin” as Prussia in1910 would list Germany in 1920. Though the country of origin is typically stated, many instances arise where a region or city is given instead of a country, like Bavaria, a major region in Southern Germany. The most prominent concern in using census records is the reliability of the reported age. Many individuals were secretive about their age or were unaware of their actual birth date.